As demand increases for varying types of applications within mobile telecommunications networks, service providers constantly upgrade their systems in order to reliably provide an expanded functionality. What was once a system designed simply for voice communication has grown into an all-purpose network access point, providing access to a myriad of applications including text messaging, multimedia streaming, and general Internet access. In order to support such applications, providers have built new networks on top of their existing voice networks. As seen in second and third generation networks, voice services must be carried over dedicated voice channels and directed toward a circuit-switched core, while other service communications are transmitted according to the internet protocol (IP) and directed toward a different, packet-switched core. This led to unique problems regarding application provision, metering and charging, and quality of experience (QoE) assurance.
Call drops and establishment failures are two of the major issues in wireless networks that impact end user experience and cause customer dissatisfaction. Metrics have been defined to track these statistics in the networks to evaluate network performance. These metrics are termed as Network Key Performance Indicators (N-KPI).
Demonstration of Network Key Performance Indicators as impacted by a particular network element is difficult to do. If a particular network element, such as for example a Radio Network Controller (RNC) of the Universal Mobile Telecommunications Services (UTMS) is situated in a test facility, then it is difficult to ensure that the test equipment which is exercising the RNC is providing conditions matching a particular customer network given the variety of network equipment that a given customer's network may present. On the other hand, once the network equipment is deployed into the customer's network the contextual conditions presented are representative, but disentangling the contributions, positive or negative, of the specific network equipment element becomes difficult due to interoperability effects.
As telecommunication systems become more complex, testing and verification of such systems also increases in complexity. In particular, more test cases are required to exercise various protocol message flows, such as for example, the flow of protocol messages between network elements to establish a call or establish a service between those elements, and in particular, various possible combinations of protocol message sequence that might occur in a real world situation. Thus it is important to be able to test the robustness of network elements to determine if the network elements can gracefully handle receiving incongruous, unexpected or invalid protocol messages. Typical protocol generators used in network test systems only generate protocol compliant messages producing success path responses to protocol messages received from network equipment under test.
In view of the foregoing, it would be desirable to provide a method to autonomously introduce failure scenarios in protocol message flows of protocol message based communications systems to test a network element in situ in a customer network. In particular, it would be desirable to provide a means by which the response of a particular network element to a plurality of problematic messaging conditions could be demonstrated under a given statistical regime.